ˆ Note that we often make a trade-off between time and space. ˆ Time complexity ˆ Space complexity. ˆ Unlike time, we can reuse memory.

Transcription

1 ˆ We use O-notation to describe the asymptotic 1 upper bound of complexity of the algorithm. ˆ So O-notation is widely used to classify algorithms by how they respond to changes in its input size. 2 ˆ Time complexity ˆ Space complexity ˆ Note that we often make a trade-off between time and space. ˆ Unlike time, we can reuse memory. 1 The asymptotic sense is that the input size n grows toward infinity. 2 Actually, there are Θ, θ, o, Ω, and ω which are used to classify algorithms. Zheng-Liang Lu 143 / 180

2 Faster Is Better? ˆ Consider a 10-year deposit account. ˆ You deposit 10, 000 TWD in the beginning of each year. ˆ Assume that the bank pays an annual interest rate r = 10% compounded annually. ˆ Determine the balance at the end of 10 years. ˆ The answer is 175, TWD. Zheng-Liang Lu 144 / 180

3 Zheng-Liang Lu 145 / 180

4 Solution 1 1 clear; clc; 2 3 r = 0.1; 4 years = 10; 5 6 s = 1; 7 for i = 1 : years s = s * (1 + r) + 1; 9 end 10 s = s * (1 + r) * 1e4 ˆ Time complexity: O(n) ˆ Space complexity: O(1) ˆ It is similar to Horner s method. Zheng-Liang Lu 146 / 180

5 Solution 2 1 clear; clc; 2 3 r = 0.1; 4 years = 10; 5 6 s = 0; 7 for i = 1 : years 8 s = s + (1 + r) ˆ i; 9 end 10 s = s * 1e4 ˆ Time complexity: O(n) ˆ Space complexity: O(1) Zheng-Liang Lu 147 / 180

6 Solution 3 1 clear; clc; 2 3 r = 0.1; 4 years = 10; 5 s = (1 + r) * ((1 + r) ˆ years - 1) / r * 1e4 ˆ Time complexity: O(1) (Why?) ˆ Space complexity: O(1) ˆ Pros: fastest and compact; Cons: not robust Zheng-Liang Lu 148 / 180

7 Real Case: Floating Interest Rate ˆ Instead of constant interest rates, assume that the sequence of floating interest rates is r = 0.1 : 0.01 : ˆ Determine the compound amount. ˆ The balance is 125, TWD. Zheng-Liang Lu 149 / 180

8 Solution 1 clear; clc; 2 3 r = 0.1 : : 0.01; 4 years = 10; 5 6 s = 1; 7 for i = 1 : years s = s * (1 + r(i)) + 1; 9 end 10 s = s * (1 + r(end)) * 1e4 ˆ Trade-off: speed and generality Zheng-Liang Lu 150 / 180

9 Why Numerical Methods? ˆ Very few problems, however, have simple and analytical solutions. ˆ Numerical methods provide computational solutions for many crucial problems, thought the correctness of the numerical methods remains a big issue. 3 ˆ Simulation techniques 4 significantly reduce the cost both in industry and science. ˆ So the simulation techniques generate huge business profit. 3 See Generation_and_propagation_of_errors. 4 See Zheng-Liang Lu 151 / 180

10 All science is dominated by the idea of approximation. Bertrand Russell ( ) Essentially, all models are wrong, but some are useful. George E. P. Box ( ) Zheng-Liang Lu 152 / 180

11 Vectorization (Revisited) 5 ˆ The built-in functions such as sqrt(x) and exp(x) automatically operate on array arguments to produce an array result with the same size as the array argument x. 1 >> t = linspace(0, pi, 5); 2 >> y = sin(t) 3 4 y = More about vectorization. Zheng-Liang Lu 153 / 180

12 Advantages by Vectorization ˆ Appearance: vectorized mathematical code appears more like the mathematical expressions found in textbooks, making the code easier to understand. ˆ Less error prone: without loops, vectorized code is often shorter. ˆ Fewer lines of code mean fewer opportunities to introduce programming errors. ˆ Performance: vectorized code often runs much faster than the corresponding code containing loops. Zheng-Liang Lu 154 / 180

13 Performance Analysis In Real Time ˆ In addition to the theoretical analysis of algorithms, programmers can also use a timer to measure the performance. 6 ˆ Once you identify which functions are consuming the most time, you should determine why you are calling them, then look for alternatives to improve the overall performance. ˆ However, Amdahl s law 7 states that the speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program. 6 Note that the results may differ depending on the difference of run-time environments, so make sure that you benchmark the algorithms on the same conditions. 7 Amdahl (1967). Zheng-Liang Lu 155 / 180

14 Digress: Amdahl s Law ˆ Assume that a program has some codes which takes α% of total running time and can be further parallelized. ˆ Then the speedup can be achieved by investing p computers. ˆ More explicitly, Speedup = ˆ So we can calculate its limit, lim p α p α p (100 α) = + (100 α) 100 α. ˆ For example, the upper bound of speedup is 2 if α = 50. Zheng-Liang Lu 156 / 180

15 tic & toc ˆ The command tic makes a stopwatch timer start. ˆ The command toc returns the elapsed time ( 10 6 second) from the stopwatch timer started by tic. (Try.) 1 >> tic % Please wait for a second. 2 >> toc 3 4 Elapsed time is seconds. Zheng-Liang Lu 157 / 180

16 Tips for Performance 8 ˆ Preallocate arrays ˆ Repeatedly resizing arrays often requires Matlab to spend extra time looking for larger contiguous blocks of memory, and then moving the array into those blocks. ˆ Vectorize your code ˆ Create new variables if data type changes ˆ Use functions instead of scripts ˆ Avoid overloading Matlab built-in functions 8 See Techniques for Improving Performance. Zheng-Liang Lu 158 / 180

17 Example: A Benchmark 1 clear; clc; 2 3 order = 0 : 1 : 4; 4 t1 = zeros(1, length(order)); 5 t2 = zeros(1, length(order)); 6 t3 = zeros(1, length(order)); 7 8 for j = 1 : length(order) 9 num = 10 ˆ order(j); tic 12 y = []; 13 for i = 1 : num 14 y = [y, i ˆ 2]; % dynamic allocation of... array y 15 end 16 t1(j) = toc; Zheng-Liang Lu 159 / 180

18 17 18 clear y; 19 tic 20 y = zeros(1, num); % preallocation of array y 21 for i = 1 : num 22 y(i) = i ˆ 2; 23 end 24 t2(j) = toc; clear y; 27 t = 1 : 1 : num; 28 tic 29 y = t.ˆ 2; % vectorization 30 t3(j) = toc; 31 end Zheng-Liang Lu 160 / 180

19 t1 / t2 t1 / t3 t2 / t Speedup Array Size Zheng-Liang Lu 161 / 180

20 t1 / t2 t1 / t3 t2 / t3 Speedup (log scale) Array Size (log scale) Zheng-Liang Lu 162 / 180

21 All roads lead to Rome. Anonymous 但如你根本並無招式, 敵人如何來破你的招式? 風清揚 笑傲江湖 第十回 傳劍 Zheng-Liang Lu 163 / 180

22 1 >> Lecture 3 2 >> 3 >> -- Functions 4 >> Zheng-Liang Lu 164 / 180

23 Functions ˆ The first thing of algorithm design is to divide and conquer. ˆ In other words, a large and complicated problem would be conquered by solving its subproblems. ˆ In order to reuse the algorithms without copying the codes, the best way is to make them functions. 9 ˆ The idea of the (software) functions is similar to the math function, which is typically written in form of y = f (x) with the input x and the output y. 9 Make bricks for your castle. Zheng-Liang Lu 165 / 180

24 ˆ A function is a piece of computer code that accepts an input argument from the caller, and returns output argument for the specific job the caller dealing with. ˆ Functions allow you to modularize a program by separating its tasks into self-contained units. ˆ We can program more efficiently and avoid rewriting the computer code for calculations that are performed frequently Remember that the bug propagates when you copy and paste the codes. It is a serious problem especially when you are working on a medium- or huge-level project. Zheng-Liang Lu 166 / 180

25 Arithmetic Functions Zheng-Liang Lu 167 / 180

26 Trigonometric Functions 11 ˆ Recall that 1 rad = 180 π. 11 See Table in Palm, p Zheng-Liang Lu 168 / 180

27 Rounding Functions ˆ Can you organize an algorithm for the function ceil and floor? ˆ How to check if the input is an integer? (Try.) Zheng-Liang Lu 169 / 180

28 Discrete Math Functions Zheng-Liang Lu 170 / 180

29 Max Functions Zheng-Liang Lu 171 / 180

30 ˆ The function min works in a similar way to max. Zheng-Liang Lu 172 / 180

31 Functions for Central Tendency Zheng-Liang Lu 173 / 180

32 Variance and Standard Deviation Zheng-Liang Lu 174 / 180

33 Functions for Sizes Zheng-Liang Lu 175 / 180

34 Random Number Generators ˆ You may use randi(n, m, n) to produce an m-by-n random integer matrix ranging from 1 to N. Zheng-Liang Lu 176 / 180

35 ˆ You can generate a random number sampled from a standard uniform distribution, say by a linear congruential generator. 12 ˆ Be aware that there is no true random number generator in the machines. 13 ˆ Widely used in Monte Carlo simulation 14 and random number generation of other distributions 15. ˆ Use rng( shuffle ) to generate different random sequences. 12 See 13 For now, the modern computers are all deterministic. Quantum computers share theoretical similarities with non-deterministic and probabilistic computers. 14 See Glasserman (2003). 15 See the acceptance-rejection method and Metropolis-Hastings algorithm. Zheng-Liang Lu 177 / 180

36 User-Defined Functions ˆ A user-defined function is created by 1 function [outputvar] = funcname(inputvar) 2 % comment section 3 end ˆ The output variables, if there exist, are enclosed in square brackets. ˆ The input variables, if there exist, must be enclosed with parentheses. ˆ funcname should start with a letter, and be the same as the file name in which it is saved. ˆ Before this function can be used, it must be saved into the current folder If not, change the current folder or add to the path pool. Zheng-Liang Lu 178 / 180

37 Example: Addition of Two Numbers 1 function z = myadd(x, y) 2 % input: x, y (two numbers) 3 % output: z (sum of x and y) 4 z = x + y; 5 end ˆ It seems bloody trivial. ˆ Actually, the plus sign is a kind of syntactic sugar Recall how to use an addition instruction in assembly codes. Zheng-Liang Lu 179 / 180

38 Example: Mean of A Sequence 1 function y = mymean(x) 2 % input: x (array) 3 % output: y (mean) 4 5 sum = 0; 6 n = length(x); 7 for i = 1 : n 8 sum = myadd(sum, x(i)); % call myadd 9 end 10 y = sum / n; 11 end Zheng-Liang Lu 180 / 180